Software Managed Manycores

Caching as a concept has been highly successful in various aspects of computer system design. Examples of cache memory abound in contemporary designs: a disk buffer cache is a small amount of buffer memory present on a hard drive to speed up the access to the disk; a web cache provides a mechanism for temporary storage of web documents to improve user experience; a DNS cache stores queried results for a period of time in the domain name system to make DNS lookup faster; P2P caching is a technique to reduce bandwidth costs for content on peer-to-peer networks; and database caching is a mechanism used to cache database content in multi-tier applications. Perhaps the earliest use of cache memory was in processor design, where caching in various forms (e.g., data caching, instruction caching, page table caching) was exploited for improving performance by reducing accesses to slower, offchip memories. Caches store frequently accessed data in a memory close to the processor to make the memory accesses faster and consume lower power. Traditionally caches in the processor have been implemented in hardware. Here the movement of the data between the caches and main memory (i.e., caching) is performed automatically by hardware—with the software oblivious to the caching mechanisms. Caching in computer architecture is typically implemented in hardware to reduce latency because each cache access is typically in the timing critical path of instruction execution. Another advantage of hardware caching is that programmers can develop software without worrying about caching since it is handled automatically in hardware. Furthermore, in order to bridge the increasing latency of memory accesses, multiple levels of caching (organized as a cache hierarchy) became popular in Fig. 2: Difference between Cache and SPM—the hardware view: SPM is just a raw memory without the hardware mechanism to manage it (as is present in caches).